{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using GPUs\n",
    "- Training on GPUs (graphic cards) makes training neural networks much faster than running on CPUs\n",
    "- Keras supports training on GPUs with both Tensorflow & Theano backend\n",
    "    - docs: https://keras.io/getting-started/faq/#how-can-i-run-keras-on-gpu\n",
    "\n",
    "\n",
    "<br>\n",
    "<img src=\"https://blogs.nvidia.com/wp-content/uploads/2016/03/titanxfordeeplearning.png\" style=\"width: 600px\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation and checkups \n",
    "- First, download and install **CUDA & CuDNN** (assuming that you are using NVIDIA gpus)\n",
    "    - Note: Installing CuDNN will enable you to use CuDNNGRU & CuDNNLSTM layers, which is about x10 times faster than GRU & LSTM layers (for more information, refer to: [CuDNN layers](https://github.com/buomsoo-kim/Easy-deep-learning-with-Keras/tree/master/3.%20RNN/4-Advanced-RNN-3))\n",
    "          - doc: https://keras.io/layers/recurrent/#cudnngru\n",
    "          - doc: https://keras.io/layers/recurrent/#cudnnlstm\n",
    "    - url: https://developer.nvidia.com/cudnn\n",
    "- Then, install **tensorflow-gpu** (gpu-enabled version of Tensorflow) by typing below in cmd or terminal\n",
    "    - pip install tensorflow-gpu\n",
    "- Then check if your machine is utilizing GPU device\n",
    "    - In my case, I have one GPU device (whose name is \"/device:GPU:0\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[name: \"/device:CPU:0\"\n",
       " device_type: \"CPU\"\n",
       " memory_limit: 268435456\n",
       " locality {\n",
       " }\n",
       " incarnation: 7806512415018627140, name: \"/device:GPU:0\"\n",
       " device_type: \"GPU\"\n",
       " locality {\n",
       "   bus_id: 1\n",
       " }\n",
       " incarnation: 11870675584284092612\n",
       " physical_device_desc: \"device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0, compute capability: 6.1\"]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.python.client import device_lib\n",
    "device_lib.list_local_devices()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 32.]]\n"
     ]
    }
   ],
   "source": [
    "# just checking if tensorflow is using GPU device\n",
    "with tf.device('/device:GPU:0'):\n",
    "    x = tf.constant([1.0, 2.0, 3.0], shape = [1,3], name = 'x')\n",
    "    y = tf.constant([4.0, 5.0, 6.0], shape = [3,1], name = 'y')\n",
    "    z = tf.matmul(x,y)\n",
    "    sess = tf.Session(config = tf.ConfigProto(allow_soft_placement = True))\n",
    "    print(sess.run(z))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
